wtclient: prune closable sessions when the tower is unreachable - #10947
Open
jkuchar wants to merge 2 commits into
Open
wtclient: prune closable sessions when the tower is unreachable#10947jkuchar wants to merge 2 commits into
jkuchar wants to merge 2 commits into
Conversation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10646
Problem
When every channel backed up by a watchtower session is closed, the session becomes "closable" and
handleClosableSessionstries to notify the tower that it may delete the session (viadeleteSessionFromTower) before removing the client's own copy.If the tower is permanently unreachable (offline, or deactivated/removed by the user), that dial fails, the loop
continues, andDB.DeleteSessionis never reached. As a result:ListClosableSessionsreloads it and the client re-dials the dead tower. With many sessions this keepslndbusy for a very long time on startup (the reporter saw "hours"), emitting a stream oferror deleting session ... from tower: failed to dial tower(...).wtdb.RemoveToweronly deletes a tower that has zero sessions — otherwise it just marks it inactive. So the dead tower's sessions, and the tower itself, are stuck forever (also the subject of [feature]: cleanup of wtclient.db / removal of watchtower #7035).Fix
Notifying the tower is only a courtesy so that it can reclaim disk space: a closable session's channels are all closed, so the local copy no longer protects anything (a breach remedy is impossible once the funding output has been spent by a confirmed close). So:
deleteSessionFromTowerfails, we log a warning and still delete the local session copy, so a dead/removed tower can no longer make closable sessions accumulate or block startup.Testing
Two new cases in
TestClient:The full
watchtower/wtclientsuite passes.Not in this PR (possible follow-ups)
--force/purge escape hatch so an operator can immediately remove a dead tower and its remaining sessions (relates to [feature]: cleanup of wtclient.db / removal of watchtower #7035).ConnectionTimeoutper address). With the two changes here a dead tower's sessions are pruned after a single failed dial rather than re-dialed forever, so this is an optional optimization.